Prioritized-LRTA*: Speeding Up Learning via Prioritized Updates
نویسندگان
چکیده
Modern computer games demand real-time simultaneous control of multiple agents. Learning real-time search, which interleaves planning and acting, allows agents to both learn from experience and respond quickly. Such algorithms require no prior knowledge of the environment and can be deployed without pre-processing. We introduce PrioritizedLRTA*, an algorithm based on Prioritized Sweeping. This novel method focuses learning on important areas of the search space. A state’s importance is determined by the magnitude of the updates made to its neighbors. Empirical tests on path-finding in commercial game maps show a substantial learning speed-up over state of the art learning real-time heuristic search algorithms.
منابع مشابه
Real-Time Heuristic Search with a Priority Queue
Learning real-time search, which interleaves planning and acting, allows agents to learn from multiple trials and respond quickly. Such algorithms require no prior knowledge of the environment and can be deployed without pre-processing. We introduce Prioritized-LRTA* (P-LRTA*), a learning real-time search algorithm based on Prioritized Sweeping. P-LRTA* focuses learning on important areas of th...
متن کاملSpeeding Up Learning in Real-time Search via Automatic State Abstraction
Situated agents which use learning real-time search are well poised to address challenges of real-time path-finding in robotic and computer game applications. They interleave a local lookahead search with movement execution, explore an initially unknown map, and converge to better paths over repeated experiences. In this paper, we first investigate how three known extensions of the most popular...
متن کاملSpeeding up the Convergence of Real-Time Search
Learning Real-Time A* (LRTA*) is a real-time search method that makes decisions fast and still converges to a shortest path when it solves the same planning task repeatedly. In this paper, we propose new methods to speed up its convergence. We show that LRTA* often converges significantly faster when it breaks ties towards successors with smallest f-values (a la A*) and even faster when it move...
متن کاملSpeeding up the Convergence of Real-Time Search: Empirical Setup and Proofs
This technical report contains the formal proofs for all of our theoretical results, as well as a description of our experimental setup for all of the results given in our AAAI-2000 paper entitled Speeding up the Convergence of Real-Time Search. In that paper, we propose to speed up the convergence of real-time search methods such as LRTA*. We show that LRTA* often converges significantly faste...
متن کاملDistributed Prioritized Experience Replay
We propose a distributed architecture for deep reinforcement learning at scale, that enables agents to learn effectively from orders of magnitude more data than previously possible. The algorithm decouples acting from learning: the actors interact with their own instances of the environment by selecting actions according to a shared neural network, and accumulate the resulting experience in a s...
متن کامل